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Abstract 

With the arrival of online survey tools that are low-cost, readily available and easy to 
administer, all organizations have access to one of the most effective mechanisms for 
determining quality improvement priorities and measuring progress towards achieving 
those priorities over time. This case study outlines the use made of this simple tool by a 
research office in one of Australia's most research-intensive universities during a substantial 
change management exercise over the period 2008-2011. The rationale for reaching out to 
the University's researchers as clients; basic principles followed to ensure high response 
rates and robust results; uses made of the data; and contribution to the change process are 
described, with a view to assisting research management professionals who are setting in 
place similar monitoring systems as an alternative to, or complementing, process-related 
(time and effort on task) performance data. 


Context 

Th e research office that is the subject 

of this case study was substantially 
restructured in 2008 with the specific and 
simple objective of improving researchers' 
satisfaction with the University's research 
management services. The objective was 


not a gratuitous one. Researchers require 
robust management systems to support 
their activities in a funding environment 
that is highly competitive and carrying a 
significant compliance burden. If they are 
not well supported, they are likely to scale 
down, or fail in, their grant seeking 
activities; funding will diminish; and there 
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is a risk that whole research programs 
could be shut down due to compliance 
breaches. An effective barometer for 
excellent research management will 
undoubtedly be how it is perceived by 
those who rely on it. 

An effective barometer 
for excellent research 
management will 
undoubtedly be how it is 
perceived by those who 
rely on it. 

Over the period covered by this case 
study, the research office managed 
significant volumes of applications 
(around 1,200 annually), grants (upwards 
of 1,500 under management), accounts 
(over 5,000 individual research accounts), 
and upwards of $200 million AUD in 
external research funding. As are all 
research offices of this size and scale, it 
was a high pressure environment where 
deadlines were externally driven, 
workloads had significant peaks at certain 
times of year, and a great deal was at 
stake for individual researchers and the 
university. 

The office's restructure represented 
the second significant restructure of 
research management delivery at the 
university in a three-year period. The first 
restructure had caused significant 
disruption to existing workflows and 
disquiet among the grants management 
staff. A quite complex matrix structure 


had been set in place, with dual reporting 
lines leading to a lack of clarity as to 
where actual responsibility for actions lay. 

The first restructure had been 
operationalized via an external 
consultancy with no continuity through 
early stage implementation. 
Unsurprisingly there was little ownership 
of the arrangements among office staff 
and external stakeholders. During one-to- 
one conversations, researchers reported 
"not knowing who to talk to any more" 
and provided numerous instances of grant 
management issues having arisen and 
been left unresolved. The unfortunate 
consequence was that the staff who could 
have made a positive difference were 
firmly entrenched in 'bunker mode'. From 
their perspective, the fault lay with the 
restructure. Past arrangements were better 
and it was unfair that the perceptions of 
poor service delivery were being seen as 
'their fault'. 

Rationale for Using the 
Online Survey Tool 

Sharon Cole, in her article 

"Reframing research administration" 
(Cole, 2010) concluded that a co-operative 
approach involving faculty and 
administrators and attention to 
organizational culture are vital 
ingredients for successful improvement in 
research services delivery. The situation 
was evidence of this and the survey tool 
was directed at both of these imperatives. 
It would provide an avenue for broad 
stakeholder input and engagement in the 
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change process and demonstrate the office 
was listening and responding in a 
proactive way. 

The 2007 restructure had adopted a 
hierarchical approach with a twist—a 
small Operations Unit was established 
alongside the customer-focused Grants 
and Ethics Teams, to ensure consistency of 
process, deal with generic activities 
(creation of 'shell' records; file 
establishment; system-driven 
communications), manage and 
continuously improve systems (including 
database management), and conduct 
quality assurance. The customer-focused 
teams, on the other hand would have a 
firm focus on researchers, with service 
and relationship-building as key drivers 
for their work. 

The structure was very different from 
anything in place in the larger Australian 
universities; it introduced high levels of 
accountability and challenged the 
formerly very autonomous management 
culture. Low morale among the staff, 
weary of change, meant that over the first 
twelve months most positions were re¬ 
filled. This led to further controversy and 
unease amongst stakeholders. 

Given the climate of discontent and 
the major change agenda, the client 
survey tool was going to provide a Tine in 
the sand'. Researchers would be able to 
rate all services provided by the office and 
remark on whether they perceived any 
improvements, as well as provide 
comments and suggestions and endorse 
particular members of staff whom they 


had found helpful over the last 12 months. 
The responses would provide the office 
with clear direction on what areas of the 
business to prioritize in order to achieve 
positive outcomes. As importantly, the 
survey would ensure that those involved 
in and impacted by the change 
management process focused on the 
relevant time period. Researchers had 
hitherto been demonstrating long 
memories—quoting poor service or an 
instance of bad practice from several years 
prior if canvassed for their opinion of the 
Office. The survey would lock down those 
observations and relegate them to history. 

All the same, the exercise was met 
with scepticism from some quarters, not 
least a number of key staff who expressed 
such reasons as 'you will never make 
some researchers happy' and 'the ones 
that like us won't respond' to argue the 
futility of the exercise. 

Basic Principles Followed 

The survey was voluntary and more 

in the nature of a conversation than a 
quantitative tool to test particular 
assumptions. All researchers who had 
applied for a research grant received an 
email invitation to provide feedback 
(rather than complete a survey) on the 
office's services, so that issues could be 
addressed and future service delivery 
could be improved. The information 
collected did not include personal 
information. However, the respondent 
was given the option of providing an 
email address if they wanted a staff 
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member to contact them to discuss an 
issue they had raised. The purpose of the 
exercise was clearly described—the office 
was committed to continuous 
improvement and feedback would be 
used to develop and improve services 
over time. People who participated would 
be informed of findings and the progress 
that was being made to address issues 
raised. 

All of the office's stakeholders, 
including other university professional 
staff, were individually contacted and 
invited to provide their views. This 
amounted to around 1,100 individuals 
who had interacted with the office in any 
one year, rising over time. In later years, a 
friendly invitation to complete the survey 
was also added to staff email signatures. 
This re-enforced the message that the 
office was looking for input on how to 
improve services, created impetus (the 
invitation was connected with their 
experience of service delivery), and led to 
much higher response rates (rising from 
just over 20% in the first survey to more 
than 35% in 2011). The consistency in the 
responses received suggested this return 
rate was adequate for good judgment at 
the outset, and more than enough by 2011. 

Only a few questions were asked. 
Respondent burden was kept low and the 
invitation alerted respondents to the fact 
that the survey would be a quick 5-minute 
exercise. Every question was neutral, with 
a view to simply covering all service 
delivery aspects. The wording of 
questions was simple and unambiguous 


so as to provide clear guidance on quality 
improvement opportunities. 

The questions were a mix of ratings 
and open-ended opportunities to 
elaborate on service delivery and never 
failed to produce a rich set of responses 
and a real sense of how the office was 
performing. The questions looked back— 
"do you think we've improved or gone 
backwards?" They also looked forward — 
"what would you like to see us change for 
next year....?" 

The ratings scale was a 6-point scale 
ranging from very high to very low, and 
did not allow respondents to 'fence sit'. 
The middle rankings had the descriptors 
'better than average' and 'worse than 
average'. This would force respondents to 
put forward a considered view even if 
they had not thought a great deal about 
the office's services in the past. The 
underlying message was 'tell us what you 
really and reasonably think we should be 
achieving'. 

The seven standard annual questions 
were neutral in the sense that they did not 
try to focus on any particular known issue 
or concern, and covered: 

• Respondent role (stage of career, 
researcher, executive, administrator, 
new to UNSWA, new to the office). 
Respondents could indicate that they 
had multiple roles. This allowed us to 
analyze the rankings by stakeholder 
group. 

• How they rated the office's services. 
This was presented as a table of 
services, from pre-award support to 
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legal and ethics clearances, provision 
of research data, communications, and 
training. Opportunity for an open- 
ended comment was provided at the 
bottom of this table. Interestingly, a 
not uncommon open-ended comment 
was "I didn't know you did all of 
those things", suggesting the survey 
was raising awareness of the 
comprehensive nature of the office's 
pre-and post-award service delivery. 

• What they liked about the office's 
services in the year 

• What they would like to see changed 
in the following year 

• Whether they thought services had 
improved, stayed the same, or gone 
backwards (or had no view) 

• Whether they would like to commend 
any particular staff members 

• Whether they would like to make any 
further comments 

Sometimes other 'omnibus' questions 
were cautiously added about particular 
management issues—for example, in 2011 
two questions were added about 
perceptions of training delivery and 
whether researchers would like to receive 
communications via new media (Twitter; 
Facebook, etc.). The importance of 
maintaining a balance between survey 
burden and opportunity to learn more 
was a foremost consideration. 

Uses Made of the Data 

Initial concerns that the exercise 

might provide a vehicle for a minority of 
disgruntled people to unreasonably 


criticize the office were quickly dispelled. 
The feedback was always remarkably 
consistent across the respondent pool and, 
as many a market research professional 
will say, they had a commonsense 'feel' to 
them, confirming in the main what staff 
already knew about service delivery 
weaknesses and providing a mandate for 
addressing issues raised. 

Those staff who expected the worst 
were pleasantly surprised to find that the 
number of people who rated their services 
on the positive side of the scale 
outnumbered those who expressed 
dissatisfaction, and that only a very few 
researchers had extremely negative views. 
What emerged was a sense that the 
office's clients were equally and 
reasonably invested in service 
improvement. They became, in a very real 
sense, part of the quality improvement 
team through having aired their views. 

What emerged was a sense 
that the Office's clients 
were equally and 
reasonably invested in 
service improvement. 

The results were reported back to 
stakeholders after analysis and the actions 
that had been taken to respond to issues 
raised formed a brief preamble to the next 
year's call for feedback. Year-on-year 
comparisons were reported to the 
Committee on Research and the Vice 
Chancellor's Advisory Council, ensuring 
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that senior management were as well 
informed as those close to the coalface, 
and aware of the continuous 
improvement efforts and achievements 
being made. Individual staff used 
examples from actions arising when 
preparing their documentation for their 
annual performance appraisal, creating 
grassroots buy-in both to hearing what 
people had to say about service delivery 
and doing something about it. 

Contribution to the Change 
process 

The results were a primary input for 

the annual research office staff retreat. As 
themes emerged, staff became active in 
contributing to addressing identified 
needs not just during the planning day 
but throughout the year. For example, the 
main concerns expressed one year were 
around an apparent lack of consistency in 
advice given at the pre-award stage. The 
grants teams responded by leading a 
comprehensive recruitment and training 
strategy for the casually appointed 
compliance advisers and introducing 
checklists and in-round debriefs that 
ensured consistency of advice. 

The following year, the scales tipped 
further into the positive and the feedback 
from researchers then focused on a desire 
for personal attention and value-added 
strategic counselling, over and above the 
(now consistently provided) basic 
compliance advice. 

Similarly, a concern regarding phone 
responsiveness was met with the 


appropriate technical and team response, 
and with due acknowledgment to the 
changing nature of office 
communications. 

Over time the only difficult-to-address 
issues became those for which the office's 
operations were reliant on the input of 
other departments. In those instances, the 
customer feedback was a primary, 
objective driver for the negotiation of 
service-level agreements that had quality 
assurance at their core. 

Commendations were passed on and 
successes from year to year celebrated. 

The office developed a continuous 
improvement culture and its stakeholders 
continued to demonstrate their 
willingness to articulate room for 
improvement, in a positive way, by 
providing constructive input and taking 
the time to provide praise where praise 
was due. 

The Office developed a 
continuous improvement 
culture and its stakeholders 
continued to demonstrate 
their willingness to articulate 
room for improvement, in a 
positive way, by providing 
constructive input and taking 
the time to provide praise 
where praise was due. 
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Future Directions 

At the time this case study was 

prepared, the office was relying primarily 
on the annual customer satisfaction 
survey and a handful of other high-level 
indicators to measure its annual service 
effectiveness. Transaction times were 
being used only in the area of ethics 
application review, where deadlines tend 
not to be set by funding agency grant 
rounds, making internal tracking more 
important. However, the electronic grants 
management system in use at the office 
provides for load reporting as well as 
transaction time reporting and the 
workflows are well-documented, allowing 
case-by-case consideration of staff 
resourcing, process effectiveness, and staff 
responsiveness. 

While the annual call for feedback 
provided an important catalyst for 
positive change and responsive service 
delivery, a next step in optimizing 
efficiency and effectiveness might lie in 
seeking improvements at a more granular 
level and taking a more comprehensive 
approach. Smith and Gronseth <2011) 
outlined a comprehensive Quality 
Management Systems approach to 
improving research administration at the 
Mayo Clinic as one way forward. Its RISE 
initiative includes guidance on structuring 
a team-driven change exercise that 
captures key performance data on system 
efficiencies that can be analyzed at regular 
intervals and such supporting initiatives 
as the designation of particular staff as 


Research Quality Coordinators who look 
for continuous improvement over time. 

... a next step in 
optimising efficiency and 
effectiveness might lie in 
seeking improvements at a 
more granular level and 
taking a more 
comprehensive approach. 

Conclusion 

The office's annual call for feedback 

using an online tool was a simple and 
effective means of addressing a number of 
pressing change imperatives. In providing 
an annual opportunity for everyone 
involved in the research enterprise to 
pause and reflect on how things had gone 
that year and to quickly and easily engage 
in a basic conversation about service 
provision, it built a culture supporting 
change. It allowed those who wanted 
change to articulate exactly what kind of 
change they were seeking, across any and 
all aspects of service. It allowed staff to 
respond and report back on how they 
were addressing issues over time. And it 
allowed those who wanted to 
acknowledge good service to do so, 'on 
the record'. It put everyone on the same 
page, and kept people focused on, and 
rewarded for, continuous improvement. 
Importantly, news of performance and 
progress was incorporated in whole-of- 
university reporting at the senior 
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executive level. Past perceptions of faculty 
as impossible to please, and 
administrators as only interested in 
getting the rules right, were able to be 


seen for what they were—a mythology 
with little foundation and without justice 
to all concerned. 
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Customer Satisfaction Survey 

Rating scale used: Very high; high; better than average; worse than average; low; very low 
n/a option provided for all questions 
Response required on all questions 

RespondentID 
CollectorlD 
StartDate 
EndDate 
IP Address 
Email Address 
First Name 
LastName 

1. Please indicate what role/s you have within the University by ticking which of the 
following applies to you. If you have multiple roles, please tick all that apply. 

This allows stratification of responses, to see how well client needs are being met at different 
stages of a research career, and amongst those who support research as well. 

• Early career researcher 

• Mid-career researcher 

• Senior researcher 

• Researcher holding an executive role 

• Researcher that arrived at the University in the past year 

• Researcher that has not used research office services 

• Research administrator 

• Other (please specify) 

2. How would you rate our services in 20XX? (Please tick one box for each area - compulsory 
to chose one.) 

Very high; high; better than average; worse than average; low; very low; n/a for each of these. 

The question is presented in table form with the various services below forming a line in the 
table and radio buttons for the rankings, which are expressed across the top. A response for 
each line is required before moving on. Note there is no 'fence sitting' allowed, 'average' is 
not an option, so that people are forced to express a positive or negative view. This makes 
them really think about what message they want to send to the office. 

• Helping you submit your grant application 

• Providing relevant, appropriate assistance and advice in administering your grant/s 

• Communicating grant deadlines and requirements 

• Opening research accounts 

• Assisting you in obtaining legal advice 

• Assisting you in obtaining ethics clearances 

• Providing information and advice on research ethics requirements 

• Providing you with high quality research data 

• Helping you use funding body application systems e.g (ARC RMS or NHMRC RGMS) 

• Sourcing timely and quality information from our website 
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Any comments regarding the services outlined above? 

Open ended response - people tend to want to comment on one or two services or 
experiences. A box 'per service' is not offered, so as to keep the survey short/respondent 
burden low. 

3. What did you like about our services in 20XX? 

Open ended response 

4. What would you like to see us change in 20XX? 

Open ended response 

5. How do you believe our services have changed in 20XX as compared with prior years? 
(Please tick one box) 

• Improved 

• Stayed the same 

• Gone backwards 

• No opinion 

6. Are there any staff members that you would like to commend for their service in 20XX? 

(drop down list of staff members) 

7. Please feel free to provide any further comments or advice 

Open ended response 

8. If you would like someone from the Office to contact you to discuss your feedback please 
provide your name and contact details here: 

Open ended response 






